Efficient schedules for the problem of optimal node visitation in acyclic stochastic digraphs
نویسنده
چکیده
Given a stochastic, acyclic, connected digraph with a single source node and a control agent that repetitively traverses this graph, each time starting from the source node, we want to define a control policy that will enable this agent to visit each of the graph terminal nodes a prespecified number of times, while minimizing the expected number of the graph traversals. We formulate this problem as a specially structured Discrete Time Markov Decision Process and we develop a series of computationally efficient and asymptotically optimal policies, by building upon the special structure of the problem and the relevant theory of suboptimal control.
منابع مشابه
THE RELATION BETWEEN TOPOLOGICAL ORDERING AND ADJACENCY MATRIX IN DIGRAPHS
In this paper the properties of node-node adjacency matrix in acyclic digraphs are considered. It is shown that topological ordering and node-node adjacency matrix are closely related. In fact, first the one to one correspondence between upper triangularity of node-node adjacency matrix and existence of directed cycles in digraphs is proved and then with this correspondence other properties of ...
متن کاملEfficient pac-learning for episodic tasks with acyclic state spaces and the optimal node visitation problem in acyclic stochastic digaphs
This paper considers the problem of computing an optimal policy for a Markov Decision Process (MDP), under lack of complete a priori knowledge of (i) the branching probability distributions determining the evolution of the process state upon the execution of the different actions, and (ii) the probability distributions characterizing the immediate rewards returned by the environment as a result...
متن کاملDynamic Multi Period Production Planning Problem with Semi Markovian Variable Cost (TECHNICAL NOTE)
This paper develops a method for solving the single product multi-period production-planning problem, in which the production and the inventory costs of each period arc concave and backlogging is not permitted. It is also assumed that the unit variable cost of the production evolves according to a continuous time Markov process. We prove that this production-planning problem can be Stated as a ...
متن کاملLongest Path in Networks of Queues in the Steady-State
Due to the importance of longest path analysis in networks of queues, we develop an analytical method for computing the steady-state distribution function of longest path in acyclic networks of queues. We assume the network consists of a number of queuing systems and each one has either one or infinite servers. The distribution function of service time is assumed to be exponential or Erlang. Fu...
متن کاملJoint pricing, inventory, and preservation decisions for deteriorating items with stochastic demand and promotional efforts
This study models a joint pricing, inventory, and preservation decision-making problem for deteriorating items subject to stochastic demand and promotional effort. The generalized price-dependent stochastic demand, time proportional deterioration, and partial backlogging rates are used to model the inventory system. The objective is to find the optimal pricing, replenishment, and preservation t...
متن کامل